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ABSTRACT 

We present observations of redshifted CO(l-O) and C0(2-l) in a field containing an overdensity of 
Lyman break galaxies (LBGs) at z = 5.12. Our Australia Telescope Compact Array observations 
were centered between two spectroscopically-confirmed z = 5.12 galaxies. We place upper limits on 
the molecular gas masses in these two galaxies of M(H2)< 1.7 x 10^" Mq and < 2.9 x 10^ Mq {2 a), 
comparable to their stellar masses. We detect an optically-faint line emitter situated between the 
two LBGs which we identify as warm molecular gas at z = 5.1245 ± 0.0001. This source, detected 
in the C0(2-l) transition but undetected in CO(l-O), has an integrated line flux of 0.106 ± 0.012 Jy 
kms~^, yielding an inferred gas mass M(H2)=(1.9±0.2) X lOi°M0. Molecular line emitters without 
detectable counterparts at optical and infrared wavelengths may be crucial tracers of structure and 
mass at high redshift. 

Subject headings: galaxies: evolution — galaxies: high-redshift 



1. INTRODUGTION 

Lyman-break galaxies have given us our first detailed 
look at star formation at high redshifts {z > 4). How- 
ever, our picture of the distant universe is limited because 
these objects are selected for strong rest-frame ultravi- 
olet emission arising from substantial unobscured star- 
bursts in the galaxies. Individually, they tell us little 
about darker baryons at the same redshift, whether the 
systems containing the baryons are intrinsically less UV- 
luminous or are obscured by dust. 

The Lyman-break gal axy (LBG) populat ion has now 
been probed from z = 2 (jSteidel et al.ll2003 ) up to z = 6 
IStanwav et al. 2003), with te ntative detections rep orted 
at redshifts as high as 2; = 10 (jBouwens et al.l l2005). The 
photometric and spectroscopic properties of LBGs evolve 
with increasing redshift. Typical LBGs at z « 5 have 
stellar masses of a few times 10^ Mq, are predominantly 
young (< 50Myr), and are sub-solar but not primor- 
dial in their metallicity (Z^0.2Zq), where these proper- 
ties are deduced from their rest-fra me optical and near - 
infrared spectral energy distribution (jVerma et al.ll2007l ). 

However something is missing in our picture of the 
universe at z > 5. The youth of Lyman-break objects 
when compared to the range of lookback times probed 
in a given survey suggests that only a small fraction of 
the overall population is undergoing a phase of strong 
unobscured star formation, and hence is detectable in 
the rest UV, at a given time. Fossil stellar populations 
in the most massive galaxies at the present time sug- 
gest that the bulk o f their stars were formed at z > 3 
(| Heavens et al.ll2003 ). but the mass assembly history of 
the universe as traced by LBGs implies that only 1% of 
the baryonic material that will eventually form today's 
galaxies was in collapsed systems at z = 5. Where is the 
other 99%? The bulk of the baryonic material at these 
high redshifts must be in either low mass systems or the 
gas phase. 



Damped Lyman-alpha systems in the spectra of dis- 
tant quasars have shown that dense neutral gas is ubiq- 
uitous at high redshift, holding a third of the HI atoms 
at z = 5 and appearin g in 10% of s ight-lines per unit 
cosmological distance (jProchaska et al. 2005). These 
sources represent reservoirs of neutral gas for star for- 
mation, yet are known to be deficient in H2 relative to 
local galaxies. Molecular gas is instead more tightly 
conce ntrated in high-density, relatively dusty regions 
(Zwa an fc Proc haska 2006). 

Securely identifying this material is challenging. At 
z « 5, the far-infrared lines emitted by cool and obscured 
material are redshifted to millimeter wavelengths, but 
not necessarily into an atmospheric window. At these ex- 
treme luminosity distances lines are faint, requiring long 
integration times to secure even a tentative detection. 
The fields of view of existing millimeter interferometers 
are small, and their correlators have, until recently, been 
limited to bandwidths of a few hundred MHz (equivalent 
to Az/z < 0.01). The resulting small volumes made sur- 
vey work at high redshift unfeasible. As a result, the few 
detections of line emission from molecular and atomic gas 
at high redshift have been towards active galactic nuclei 
- quasars to z = 6.4 (iBertoldi et al.ll2003l ). radio galax- 
ies to z = 5.2 amer et al.ll2005[ ) - and hence probed 
highly atypical environments. 

In this paper we present the results of a pilot pro- 
gramme to explore the cool gas associated with a large 
scale structure at z > 5 marked out by Lyman-break 
galaxies rather than by an active galaxy. Our 40 arcmin^ 
target field contains seven UV-luminous sources with 
spectroscopically confirmed redshifts in a narrow range 
(Az < 0.1), a 6(7 excess over the typical density of such 
sources in our survey, but shows no clear spatial cluster- 
ing^ 



''^ Given the redshift dispersion and spatial distribu- 



^ The detailed properties of this field, and of others probed in 
our ESO Remote Galaxy Survey (ERGS) will bo discussed in a 
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tion of these galaxies, it is clear that their star forma- 
tion cannot be a response to a common triggering event, 
and yet the probability of such a configuration arising by 
chance is small. The simplest explanation is that there 
is a large mass of hidden baryons in this field at this 
redshift. At any one time this structure is traced by 
the small fraction of that material undergoing compara- 
tively short-term UV-luminous star formation events and 
appearing as LBGs. While the stars within the LBGs 
are likely to end up in the most massive current-day 
galaxies after subsequent mergers, the total stellar mass 
in even this overdensity of LBGs is an order of magni- 
tude too small to account for the stars in the most mas- 
sive spheroids. Consequently, there should be far more 
baryons in the vicinity of the LBGs than revealed in the 
UV. Our pilot programme was designed to probe the cool 
gas mass in a small section of the structure, centered on a 
pair of LBGs with Lyman-a emission redshifts separated 
by Az = 0.004. The observations target not only the two 
LBGs, but any darker/fainter systems in the underlying 
larg e scale structure. T he results for lower redshift LBGs 
(e.g. lBaker et al.ir2004D . suggested that we were unlikely 
to detect the two known galaxies directly. 

All magnitudes in this paper are quoted in the AB sys- 
tem (' Oke fc Gunnlll983l ). We adopt a ACDM cosmology 
with (f^A, ^M, /i) = (0.7, 0.3, 0.7). 

2. OBSERVATIONS AT THE ATCA 

The observations were carried out at the Australia 
Telescope Compact Array in March- April 2008. The ar- 
ray was in the relatively compact H168 configuration, 
with both North-South and East- West baselines. We 
used the correlator in the 128 MHz/64 channel configu- 
ration with two overlapping intermediate frequency (IF) 
bands, giving a total bandwidth of 240MHz, and measur- 
ing the total incident flux density, but not polarization. 

We investigated two transitions of CO at high redshift. 
In order to constrain the CO(l-O) line, we tuned the IF 
bands to 18.838 GHz and 18.958 GHz (allowing detection 
of an emission line in the range z = 5.06 — 5.14). In 
this configuration, the spectral resolution is 4 MHz or 
63.5 km s~^. Observations were taken in three 8hr ob- 
serving periods on the nights of 2008 March 22-24 (al- 
though data taken on the 23rd were discarded due to 
poor weather). A single pointing was observed for the en- 
tire observing period, and a nearby bright source (1045- 
188) observed every 15 mins to determine phase stabil- 
ity. Pointing accuracy was checked every hour. Absolute 
flux calibration was through observations of 1921-293 
(the standard flux calibrator at the ATCA) each night. 
The half-power-beam-width (HPBW) of the ATCA at 
this frequency is 2.5 arcmin and the naturally-weighted 
restoring beam in this configuration is 15.9" x 10.9". 

The CO (2-1) line was observed in four llhr observa- 
tions on the nights of 2008 Apr 04-07 using the recently- 
commissioned 7mm receiver system. The IF bands were 
tuned to 37.672 GHz and 37.792 GHz (giving redshift cov- 
erage z = 5.09 — 5.13). A single pointing centered on 
the 18.8 GHz field center was observed, and 1045-188 
was used as a pointing and phase calibrator. Uranus, 
the primary flux calibrator for the ATCA at millimeter 
wavelengths, was not visible at any point in our observ- 
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ing period. Instead we bootstrap our primary flux cal- 
ibration from two compact HII regions, observed each 
night, which were also observed using our configuration 
by ATCA staff on the afternoon of 2008 Apr 04 together 
with Uranus and a bandpass calibrator. At this fre- 
quency, the HPBW is 74" and the restoring beam is 
7.3"x 4.8". 

The total useful observing time was 16 hours at 
18.9 GHz and 44 hours at 37.7 GHz. The RMS noise in 
the final images is 0.11 mJy/beam and 0.10 mJy/beam 
in each 4 MHz channel at 18.9 and 37.7 GHz respectively. 

3. LIMITS ON THE CO IN LYMAN BREAK GALAXIES 

As illustrated in figure [U our pointing included two 
Lyman-a emitting, Lyman-break galaxies within the 
HPBW at 18.9GHz (table [1|). At 37.7 GHz one of these 
(at z = 5.116) lies well outside the beam half-power 
point, while the second (at z = 5.120) remains close to 
the pointing center. Neither galaxy shows evidence for 
line or continuum emission in either of the two transitions 
surveyed. This is not, in itself, surprising gi ven the youth 
and low masses (typically a few xIO^Mq, IVerma et al.l 
l2007f) of z = 5 Lyman-break galaxies. However, these 
observations provide the strongest constraint yet on the 
molecular gas content of z > 5 star-forming galaxies. 

We use our limiting flux at the frequency of 
the redshifted CO (1-0) line to constrain the molec- 
ular gas content of these galaxies. We employ the 
widely-used conversion factor of Mg^s / L'^q—^ ^© 
(K k ms~^ pc^)~^ based on loca.1 infr ared-luminous galax- 
ies (|Solomon fc Vanden Bo^ |2005l ). Our image RMS 
noise of 0.11 mJy/beam at the CO(l-O) line corresponds 
to a formal 2 sigma limit L^q < 2.2 x 10^" Kkms~^ pc^ 
for the luminosity of an unresolved 200 km emission 
line at z = 5.12. Thus we constrain the molecular gas 
mass of the known Lyman-break galaxies in this field to 
be M(H2)< 1.7 x 1O1°M0 {2a). 

One of our LBGs (source B in table [1]) is also close 
to the center of the beam in our 37 GHz observations. 
For thermalized, optically-thick CO emission, the line 
luminosity is independent of transition. Making this as- 
sumption, non-detection of the 2-1 line places a tighter 
constraint on the mass of M(H2)< 2.9 x 10^ Mq (2(t), 
comparable to the inferred stellar mass in such sources. 

4. AN OPTICALLY-FAINT MOLECULAR LINE EMITTER 

Inspection of the ATCA data cubes revealed no ev- 
idence for continuum sources. Only one emission fea- 
ture was found to extend over more than two adjacent 
frequency channels. This feature, a spatially-unresolved 
emission line centered at 37.642 GHz with a FWHM of 
llOkms"^ (equivalent to four independent frequency 
channels), lies on the beam half-power point. It has a 
peak flux of 0.94 ± 0.15mJy/beam, and a velocity in- 
tegrated flux of 0.106 ± 0.012 Jy km s~^. A combined 
frame of channels not containing line emission (a total 
bandwidth of 180 MHz) shows no significant continuum 
emission with a formal limit on the continuum of <0.49 
mJy/beam at 37.73 GHz. The line emission is centered 
1 arcminute (370 kpc, comoving) north-west and 40 arc- 
seconds (250 kpc) north-east of our spectroscopically con- 
firmed Lyman-break galaxies at z — 5.120 and z = 5.116 
respectively, towards more LBGs in the field. 

This line emitter is undetected at 18.9 GHz and not 
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TABLE 1 

HlGH-2 SOURCES IN OUR TARGET FIELD. 

Source RA fc Declination (J2000) z Iab gpoAi; M(H2) 

X 10:40:46.66 -11:58:33.8 O20 27.00±0.57 '- <T?7 

B 10:40:41.51 -11:58:21.6 5.116 25.71±0.21 - < 0.29 
C 10:40:43.50 -11:57:56.8 5.125 > 26.63 0.106 ±0.012 1.9 ±0.2 

Note. — Sources A and B are spectroscopically confirmed Lyman-break galaxies, while source C is the millimeter line-emitter discussed 

in section|4] Sco^''^ is in units of Jy km s~^ and M(H2) in units of 10^" Mq as discussed in the text. 2 a limits are given where appropriate. 
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Fig. 1. — Contour map of 6-channel multi-frequency synthesis 
image centered at 37.64 GHz, overlayed on an /-band image of the 
same region. The positions of the two Lyman break galaxies at 2 = 
5.120 and z = 5.116 respectively are shown as small circles. The 
half power point of the synthesized beam at 37.7 GHz is indicated 
by a large circle. The HPBW at 18.9 GHz is twice as large and 
encompasses the whole image. Contours mark formal 2.5, 5 and 
7.5 (T surface brightness levels. Negative contours are shown with 
dashed lines. Although the right-hand LBG is situated within a few 
arcseconds of a 2.5 it contour, this is part of an extended low-level 
artifact. 

spatially coincident with a known source at any observed 
wavelength (from 0.5/im to 21cm). Our optical and IR 
data require that the host galaxy must have a flux den- 
sity < 0.2 /iJy at an observed frame wavelength of 4.5/im 
(0.7/im rest if at z = 5.125) and < 0.08 /iJy at a 0.8/xm 
(ISOOA rest). No sub -mm data exist at this location. 
We note that the lensing properties of this field are well 
under stood, and we use the lensing maps of Clow e et al.l 
((2006h to estimate a maximum 10% enhancement in the 
flux of a source at this location. 

In order to verify the reality of the 37.6 GHz line, 
we investigated its properties thoroughly. The velocity- 
integrated line is a formal 8 a detection in a multi- 
frequency synthesis image of 6 channels centered on the 
line emission. It is detected in the ww-dataset on multiple 
baselines, and in two independent sub-sets of the data. 
Its strength in the flux-calibrated image is independent 
of image deconvolution algorithm and the location of the 
imaging center. We conclude that this is unlikely to be 
a spurious detection. 

We evaluate the probability that this is a line emit- 
ter at a redshift unrelated to our high redshift system 
(i.e. not a C0(2-l) emission line), by considering the 
population models of Blain et al.l (|2000t ). These authors 
calculated the expected number density of line emitters 
as a function of frequency and line flux, given models for 
galaxy and dark matter distributions at z < 10. As a 
cautious assumption, we consider the source counts pre- 
dicted at 50 GHz at our integrated flux density (while 
noting that source counts at 37.6 GHz are likely to be 
lower). In our 256 MHz total bandwidth, and allowing 
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Fig. 2. — The spectrum of our line-emitter source. Measurements 
at 18.8 GHz (probing the CO(l-O) transition at z = 5.12) are shown 
with dotted lines, while measurements at 37.6 GHz are shown with 
solid lines. Typical error bars are shown for reference. 

for source detections within 1.5 HPBW, we would expect 
0.003 line emitters in our survey, with these dominated 
by redshifted CO (1-0) lines at z — 2.1. Coupled with 
the known existence of structure a,t z — 5.12 in this fleld, 
wc interpret the line detection as C0(2-l) emission at 
z = 5.1245 ± 0.0001 at high confidence. 

At the measured line flux in the C0(2-l) transition, 
and assuming constant luminosity, the expected counter- 
part in the 1-0 transition would be below the 2 a level at 
18.8 GHz, consistent with our non-detection. Using the 
same CO-to-H2 conversion factor discussed in section [3] 
applied to the luminosity of the C0(2-l) transition, we 
infer a molecular gas mass of M(H2)= (1.9 ± 0.2) x 10^° 
Mq in this source. 

The beam width at 37.7 GHz implies an upper limit on 
the physical size of our unresolved source of 45x30kpc. 
A simple virial calculation assuming a uniform mass dis- 
tribution suggests that our derived gas mass could rep- 
resent up to 50% of the dynamical mass in the system, 
while remaining both bound and smaller than our beam 



5. LARGE SCALE STRUCTURE AT Z = 5.12 

This pilot survey at 37.7 GHz probed only 3% of the 
projected spatial distribution and 35% of the redshift 
range occupied by clustered UV-luminous galaxies in our 
target field. Despite this, the gas mass inferred from our 
single detection is comparable to the total stellar mass in 
UV-luminous sources in the structure. If this detection is 
typical of the space density of molecular line emitters in 
this fleld, then galaxies with unobscured star formation 
represent at bes t a few percent of the baryons present 
(consistent with lVerma et aLll2007f) . 
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In the nomenclature of ISolomon fc Vanden BoiIB 
our detected source is an Early Universe Molec- 
ular Emission Line Galaxy (EMG) , a broad category in- 
cluding radio galaxies, sub-mm galaxies and QSOs at 
z > 2. It represents 10^ Mq of CO, formed within 1.1 Gyr 
of the Big Bang. This yield is plausible from 3 x 10^ 
population III stars, e ach of lOOMfri, in as lit tle as the 
stellar hfetime (10 Mvr. lHeger fc Wooslevll2002D . By con- 
trast, the yield of more conventional supernovae and the 
typical star formation rates of UV-luminous sources in 
this field (^30M ( 7)yr~ ^), 10^ years would be required 
(see IWalter et al.l 120031 ) implying either early and con- 
tinuous star formation, or an early, intense starburst. 
A source with unobscured star formation of 10 M0 yr~^ 
at z = 5.12 would have been detected in our deep op- 
tical imaging, but observations of dust emission in the 
rest-frame far-infrared will be necessary to distinguish 
between a mature stellar population at the time of obser- 
vation and ongoing, but dust-obscured, star formation. 

It is interesting to compare our detection with a sim- 
ilar observati o n at z = 5.2, also using the ATCA. 
iKlamer et al.l (|2005[ ) studied a high mass system at 
this redshift, marked out by the presence of at least 
six Lyman-a emitter galaxies and a powerful radio 
galaxy. Based on UV-luminous stellar mass, this sys- 
tem should be comp arable in mass to our target region. 
IKlamer et aH (|2005( ) detected CO emission in the 1-0 and 
5-4 transitions, clearly associated with the radio galaxy. 
While the inferred gas mass is within a factor of four of 
our object, the velocity structure differs: the emission 
line detected by Klamer et al. (2005) was twice as broad 
as our line emitter, possibly due to momentum input 
from the radio source. 

We note that the CO luminosity and derived mass of 
our line-emitter are similar to those of sub-millimeter 
galaxies (SMGs) at lower redshifts (Greve et al.. ,2005.) . 
These are also often a ssoc iated with galaxy overden- 
sities at high redshift (e.g. ISmail et alJl2003D . While 
some SMGs with this mass would have been detected 
in our deep Spzteer/IRAC imaging (based on their K- 
band magnitudes at z ~ 2.5, Small et al. 2004, rcdshiftcd 
to z = 5.12), others would not. CO emission is not 
a reliable star formation rate (SFR) indicator, but is 
loosely correlated with far-infrared luminosity, and hence 
SFR. Data on this relation at high redshift is sparse, 
but known EMGs with comparable gas masses to our 
detected source have SFRs of ^ lOOOMQyr"^. By con- 
trast, the tight limit on II2 mass in the LBGs in our field 
would imply obscured star formation rates of less than 
a few hundred solar masses per year, even given dust 
properties similar to those of SMGs. Our detected source 
likely represents a more extreme environment than that 
seen in LBGs at the same redshift and could be a high 
redshift counterpart for the intense starbursts in SMGs. 

It is becoming increasingly clear that no one sur- 
vey technique can provide a complete picture of large 
scale structure at high redshift. Overdensities of Ly- 
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man break galaxies are correlated with the presence 
of Lyman-alp ha emitters, radio galaxies, SMGs, DLAs 
(jCookeet al.ll2006D . and now molecular emission line 
galaxies at the same redshift. Each component of the 
structure may account for only a few percent of the bary- 
onic mass and each presents observational challenges. 
While LBGs and DLAs pr obe comparable mass haloes at 
z = 3 (jCooke et al.ll2006l ). molecular line emitters may 
probe more massive and denser regions of the same struc- 
tures. 

Extensive extrapolation based on a single line from 
a single source is risky. Observations of this source in 
other emission lines will be required to confirm its red- 
shift and luminosity. Such observations will also begin 
to characterize the gas temperature and density, and de- 
termine whether the CO to gas mass conversion fac- 
tor for ULIRGs is appropriate for low mass systems 
at early times. Additional sources must be identified 
and characterized to determine whether our initial de- 
tection is anomalous or typical of its population. Ulti- 
mately, more extensive millimeter surveys, based on ex- 
isting deep datasets, are required to estabhsh the molec- 
ular gas content of large scale structures at high redshift, 
and to probe this early stage of galaxy formation. 

6. CONCLUSIONS 

Our main conclusions can be summarized as follows: 

(i) We have investigated the molecular gas content, as 
traced by low-order transitions of CO, in a large scale 
structure identified in Lyman break galaxies at z — 5.12. 

(ii) Neither of two known Lyman break galaxies in our 
survey area show associated molecular line emission, sug- 
gesting that their molecular gas content does not signif- 
icantly exceed their stellar content. 

(iii) We have identified a source with line emission at 
37.642 GHz, which we identify as emission in the C0(2-l) 
transition at z = 5.1245±0.0001. This source is currently 
undetected at any other wavelength. 

(iv) The inferred gas mass in this single source is 
M(H2)= (1.9 ± 0.2) X 10^° Mq, comparable to the total 
stellar mass of UV-luminous sources in the same region. 

(v) Further studies of UV-faint sources at high red- 
shift are essential to characterize such systems, and could 
make substantial progress towards balancing the baryon 
budget at early times. 
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